This invention is directed to making predictions and, more particularly, computer-implementable methods of making predictions.
Before the development of the Internet, magazines, newspapers, radio, television, catalogs, product shows, and the like were the traditional mediums used to advertise products and services. Advertising utilizing these mediums is typically based on the content of the medium. For example, garden products are typically advertised in a gardening magazine, or at a garden show. Merchants may also send catalogs directly to consumers. Typically, catalogs are sent to consumers who may have purchased a product from the merchant or have purchased a product from a competing merchant. The purpose of advertising, of course, is to place information about a product or service before a likely buyer of the advertised product or service.
In the past, determining in advance whether an advertising campaign is likely to be successful has been limited. While, in some instances, sample advertising campaigns have been conducted before a full advertising campaign has been undertaken, this is more the exception than the rule due to product marketing pressure. Further, it may be impossible to conduct a sample advertising campaign when the advertising medium is national, such as a national magazine or television show. While after-the-fact analyses of the results of an advertising campaign have been conducted in order to improve future advertising campaigns, the cost of the current advertising campaign is lost. Obviously, this is undesirable, particularly if the advertising campaign is unsuccessful. The Internet has the capability of reducing the cost of advertisements by reducing the likelihood that an advertising campaign will be unsuccessful.
Internet users typically view banner advertisements on the World Wide Web (xe2x80x9cWebxe2x80x9d) page they are viewing. These banner ads may either be based on the content of the page they are viewing, or may be based on a profile of the consumer viewing the Web page. As is well known to those familiar with the Internet, consumer profile data is regularly collected both knowingly and unknowingly from consumers when they are connected to the Internet. Some companies ask consumers for profile data, such as age, sex, occupation, nationality, etc. Some of the same companies and other companies collect consumer profile data using xe2x80x9ccookies.xe2x80x9d As is well known to those familiar with the operation of the Internet and the Web, a cookie is a block of data that a Web server stores on a client computer. When a user returns to the same Web site, the browser operating on the client computer sends a copy of the cookie back to the server. Cookies are used to identify users, to instruct a server to send a customized version of a requested Web page, to submit account information for the user, and for other administrative purposes. Thus, cookies allow user profiles to include data such as Web sites visited and past purchases, for example.
The collection of user profile data allows Web pages to be personalized for every user. Therefore, a Web site may send different Web pages to each user, the Web page being designed to enhance the productivity or experience of the user. Personalizing a Web page for different users is intended to direct individual users to the products or services that are of likely interest to that user and that the user may not have accessed without the personalization. The problem, however, is that while Web pages or advertisements may be personalized for users, in the past, there has been no way to accurately predict how users will react to new products or what products the users are likely to purchase and, thus, predict whether an advertising campaign directed to users having a specific profile will be successful.
Many different tools have been developed to predict and identify subpopulations likely to act in a particular manner in response to a particular stimulus. For example, discriminant function analysis has been used to identify subpopulations likely to act in a particular manner in response to particular stimuli. Most discriminate functions are linear or quadratic because they are based on the assumption that the underlying data are normally distributed. However, nonparametric approaches also have been developed. In subpopulation studies the goal is not only to identify an individual for targeting, but to estimate the proportions of intermingled subpopulations that are likely to act in a particular manner in response to particular stimuli.
Unfortunately, none of the previously developed predictive tools (hereinafter called inference engines) are best under all Internet advertising situations. In a particular situation, one inference engine may be better than another. Moreover, there is no presently known way of predicting in advance which inference engine is likely to be better than others in a particular Internet advertising situation.
The present invention is directed to providing a computer-implemented method of selecting an inference engine that is best designed to meet a particular objective, and then using the selected engine to make predictions and forecast reports. While designed for use in connection with Internet advertising and describing a connection with the Internet, it is to be understood that the invention may also find use in other environments.
As briefly noted above, statisticians have demonstrated that many different analyses can be used to identify subpopulations from a mixed population of individuals. Historically, discriminant functions have been used, many based upon the assumption that the underlying data are normally distributed: namely, linear or quadratic discriminant functions. However, when considering subpopulations of customers or prospects in the business world, it is likely that data will not be distributed normally; hence, for precise identification of a customer or prospect subpopulation, a more expansive approach is needed. At a minimum this problem should be addressed by employing nonparametric approaches, which can be implemented without particular regard to the underlying distributions, together with parametric approaches. As will be better understood from the following summary, the present invention is directed to providing a unified architectural approach with a Bayes decision rule that allows any method to be cast in the same context. This allows direct comparison of algorithms, or inference engines, and immediate deployment of the best approach. Additionally, embodiments of the invention allow sample size requirements to be reduced through employment of a xe2x80x9cleaving-n-outxe2x80x9d approach. In essence, the invention has the capability of maximizing an arbitrary objective function.
The goal of the invention is not only to identify an individual for targeting, but to estimate the proportions of selected subpopulations in a larger population. This allows comprehensive performance forecasts to be made. For example, the response rate and profit for a list of targeted individuals can be estimated. The invention employs estimated density functions together with a decision array correction procedure to estimate these proportions from a set of linear relationships. The estimated density functions and the decision array that are determined as best at achieving a desired objective are persisted.
In accordance with this invention, a computer-implementable method of selecting which inference engine of a plurality of inference engines to use to create an estimated density function and a decision array for predicting the categories into which individuals fall, such as an Internet buyer/non-buyer, and produce forecast reports based on the predictions is provided. The computer-implementable method comprises a training process and an unknown sample data analysis process. The training process employs training (known) sample data that categorizes individuals based on the individuals"" profile features. The training sample data is sequentially applied to multiple inference engines to determine which engine is best based on a desired objective. Preferably, each inference engine is tested several times using different sets of individual profile features. A decision array and estimated density function associated with the best engine and the best set of features is then used during the unknown sample data analysis process to create predictions and forecast reports based on the predictions.
In accordance with other aspects of this invention, the training process involves identifying a data source for each category, establishing a training sample set and creating and storing a training data structure. After the training data structure is created and stored, the data structure is analyzed.
In accordance with further aspects of this invention, the training data structure is analyzed using a leaving-n-out approach. First a category is selected. Next, n selected individuals"" related data is removed from the training data structure. Then, a density function is estimated and a density value is calculated for each category based on the training data structure with the selected individual""s data removed. The selected individual""s data is reinserted into the training data structure and the foregoing sequence is repeated for another individual. After a density function has been estimated and a density value calculated for each category for each individual with the individual""s data removed, the entire sequence is repeated for the next category. Processing continues until all categories have been analyzed. The value of n can vary from 1 up to half of the individuals comprising the training data structure.
In accordance with still further aspects of this invention, the calculated density values are used to create a density value data structure. Then, for each category and each individual, a decision rule is applied to the density value data structure for the individual. The results are used to create a decision array. After completion, the decision array is displayed so that a user can determine if the objective has been met.
In accordance with further aspects of this invention, if the objective is changed by the user, the objective may be reset and the training process repeated.
In accordance with still other aspects of this invention, if the objective has been met, the decision array and the estimated density function are stored.
In accordance with yet other aspects of this invention, an engine selection is made based on which engine""s results best achieve the desired objective.
In accordance with yet still other aspects of this invention, the decision array and estimated density function associated with the selected engine are applied to an unknown sample data structure created from the unknown sample data to obtain prediction data that is used to create forecast reports.
As will be readily appreciated from the foregoing summary, the invention provides a computer-implemented method of selecting the best inference engine to meet a desired objective. The invention is ideally suited for use in forecasting the likelihood that particular Internet advertising campaign will be successful, i.e., determining if a product or service to be advertised will be purchased by a sufficient number of individuals whose Web page is personalized to receive advertising information about the product or service. In addition to buyers/non-buyers, the invention can be utilized to predict other types of responses such as clicker vs. skipper, respondent vs. non-respondent. The invention can also be utilized in other environments to predict results, such as sick vs. healthy (medical diagnosis), or friend v. foe (weapons systems). Generally speaking, particularly when applied to the Internet, the invention can be used to predict the proportions of a subpopulation in a larger population likely to respond to a particular stimulus in a particular manner.